Using Split Samples and Evidence Factors in an Observational Study of Neonatal Outcomes

observational study of neonatal outcomes Kai Zhang, Dylan Small, Scott Lorch, Sindhu Srinivas, Paul R. Rosenbaum1 University of Pennsylvania, Philadelphia Abstract. During a few years around the turn of the millennium, a series of local hospitals in Philadelphia closed their obstetrics units, with the consequence that many mothers-to-be arrived unexpectedly at the city’s large, regional teaching hospitals whose obstetrics units remained open. Nothing comparable happened in other US cities, where there were only sporadic changes in the availability of obstetrics units. What effect did these closures have on mothers and their newborns? We study this question by comparing Philadelphia before and after the closures to a control Philadelphia constructed from elsewhere in Pennsylvania, California and Missouri, matching mothers for 59 observed covariates including year of birth. The analysis focuses on the period 1995-1996, when there were no closures, and the period 1997-1999 when five hospitals abruptly closed their obstetrics units. Using a new sensitivity analysis for difference-indifferences with binary outcomes, we examine the possibility that Philadelphia mothers differed from control mothers in terms of some covariate not measured, and perhaps the distribution of that unobserved covariate changed in a different way in Philadelphia and control-Philadelphia in the years before and after the closures. We illustrate two recently proposed techniques for the design and analysis of observational studies, namely split samples and evidence factors. To boost insensitivity to unmeasured bias, we drew a small random planning sample of about 26,000 mothers in 13,000 pairs and used them to frame hypotheses that promised to be less Address for correspondence: Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104-6340 USA. This work was supported by grants SES-0849370 and SES-0961971 from the U.S. National Science Foundation. Kai Zhang is a PhD student, Dylan Small is associate professor, and Paul R. Rosenbaum is professor in the department of statistics at the Wharton School of the University of Pennsylvania. Scott Lorch is assistant professor of pediatrics and Sindhu Srinivas is assistant professor of obstetrics and gynecology at the University of Pennsylvania School of Medicine. 5 April 2011